Revisiting the Spatial and Temporal Modeling for Few-Shot Action Recognition

Spatial and temporal modeling is one of the most core aspects few-shot action recognition. Most previous works mainly focus on long-term relation based high-level spatial representations, without considering crucial low-level features short-term relations. Actually, former feature could bring rich local semantic information, latter represent motion characteristics adjacent frames, respectively. In this paper, we propose SloshNet, a new framework that revisits for recognition in finer manner. First, to exploit features, design fusion architecture search module automatically best combination features. Next, inspired by recent transformer, introduce model global relations extracted appearance Meanwhile, another encode between frame representations. After that, final predictions can be obtained feeding embedded spatial-temporal common frame-level class prototype matcher. We extensively validate proposed SloshNet four datasets, including Something-Something V2, Kinetics, UCF101, HMDB51. It achieves favorable results against state-of-the-art methods all datasets.